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Security,  House  of  Representatives 


Accountability-  Integrity  Reliability 

Highlights 


Why  GAO  Did  This  Study 


What  GAO  Found 


The  Department  of  Homeland 
Security  (DHS)  has  relied  on 
service  acquisitions  to  meet  its 
expansive  mission.  In  fiscal  year 
2006,  DHS  spent  $12.7  billion  to 
procure  services.  To  improve 
service  acquisition  outcomes, 
federal  procurement  policy 
establishes  a  preference  for  a 
performance-based  approach, 
which  focuses  on  developing 
measurable  outcomes  rather  than 
prescribing  how  contractors  should 
perform  services. 

This  testimony  focuses  on  how 
contract  outcomes  are  influenced 
by  how  well  DHS  components  have 
defined  and  developed  contract 
requirements  and  performance 
standards,  as  well  as  the  need  for 
improved  assessment  and  oversight 
to  ensure  better  acquisition 
outcomes. 

GAO’s  statement  is  based  on  its 
report  being  released  today,  which 
reviewed  judgmentally  selected 
contracts  for  eight  major 
investments  at  three  DHS 
components — the  Coast  Guard, 
Customs  and  Border  Protection 
(CBP),  and  the  Transportation 
Security  Administration  (TSA) — 
totaling  $1.53  billion  in  fiscal  years 
2005  and  2006;  prior  GAO  and  DHS 
Inspector  General  reviews; 
management  documents  and  plans; 
and  related  data,  including  138 
additional  contracts,  primarily  for 
basic  services  from  the  Coast 
Guard,  CBP,  TSA,  and  Immigration 
and  Customs  Enforcement. 


Over  the  past  several  years,  GAO  has  found  that  appropriate  planning, 
structuring,  and  monitoring  of  agency  service  acquisitions,  including  those 
that  are  performance-based,  can  help  minimize  the  risk  of  cost  overruns, 
delayed  delivery,  and  unacceptably  quality.  Several  prior  GAO  and  DHS 
Inspector  General  reviews  of  major  DHS  investments  using  a  performance- 
based  approach  point  to  such  shortcomings.  While  all  of  the  contracts  GAO 
reviewed  at  the  Coast  Guard,  CBP,  and  TSA  had  outcome-oriented 
requirements,  contracts  for  four  of  the  eight  investments  did  not  have  well- 
defined  requirements,  or  a  complete  set  of  measurable  performance 
standards,  or  both  at  the  time  of  contract  award  or  start  of  work.  These 
service  contracts  experienced  cost  overruns,  schedule  delays,  or  did  not 
otherwise  meet  performance  expectations.  In  contrast,  contracts  for  the  other 
four  investments  had  well-defined  requirements  linked  to  measurable 
performance  standards  and  met  the  standards  for  contracts  that  had  begun 
work. 

In  managing  its  service  acquisitions,  including  those  that  are  performance- 
based,  DHS  has  faced  oversight  challenges  that  have  limited  its  visibility  over 
service  acquisitions  and  its  ability  to  make  informed  acquisition  management 
decisions.  Notably,  the  department  lacks  reliable  data  on  performance-based 
service  acquisitions.  About  half  of  the  138  contracts  identified  by  DHS  as 
performance-based  had  none  of  the  elements  DHS  requires  for  such  contracts: 
a  performance  work  statement,  measurable  performance  standards,  or  a 
quality  assurance  surveillance  plan.  Such  inaccurate  data  limit  DHS’s  ability  to 
perform  management  assessments  of  these  acquisitions.  In  addition,  the  Chief 
Procurement  Officer,  who  is  responsible  for  departmentwide  procurement 
oversight,  has  not  conducted  management  assessments  of  performance-based 
service  acquisitions. 

To  help  DHS  improve  outcomes  for  its  service  acquisitions,  including  those 
that  are  performance-based,  GAO  recommended  that  DHS  routinely  assess 
requirements  for  complex  investments  to  ensure  that  they  are  well-defined, 
and  develop  consistently  measurable  performance  standards  linked  to  those 
requirements.  GAO  also  recommended  that  DHS  systematically  evaluate  the 
outcomes  of  major  investments  and  relevant  contracting  methods  and 
improve  the  quality  of  data  to  facilitate  identifying  and  assessing  the  use  of 
various  contracting  methods.  DHS  generally  concurred  with  GAO’s 
recommendations,  noting  some  departmental  initiatives  to  improve 
acquisition  management. 


To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  GAO-08-765T. 

For  more  information,  contact  John  Hutton  at 
(202)512-4841  orhuttonj@gao.gov. 

_ United  States  Government  Accountability  Office 


Mr.  Chairman  and  Members  of  the  Committee: 


Thank  you  for  inviting  me  here  today  to  discuss  the  Department  of 
Homeland  Security’s  (DHS)  planning  and  assessment  of  its  complex 
service  acquisitions.  To  meet  its  expansive  homeland  security  mission, 
DHS  spends  billions  of  dollars  on  service  acquisitions  for  critical  trade, 
transportation,  and  border  security  investments.  In  fact,  more  than  80 
percent  of  DHS’s  total  procurement  dollars  are  spent  on  services.  Prior 
GAO  work  has  found  that  appropriate  planning,  structuring,  and 
monitoring  of  acquisitions  is  critical  to  ensuring  that  the  services  provided 
meet  the  government’s  needs.1  To  help  improve  service  acquisition 
outcomes,  federal  procurement  policy  calls  for  agencies  to  use  a 
performance-based  approach  to  the  maximum  extent  practicable.  This 
approach  includes  a  performance  work  statement  that  describes  outcome- 
oriented  requirements,  measurable  performance  standards,  and  quality 
assurance  surveillance.  If  properly  implemented,  these  characteristics  can 
help  ensure  that  contracted  services  meet  cost,  schedule,  and  performance 
requirements.  Other  factors,  such  as  pressure  to  get  programs  up  and 
running,  additional  external  requirements,  and  technological  challenges 
also  impact  the  ability  to  achieve  good  acquisition  outcomes. 

While  a  performance-based  approach  has  been  widely  accepted,  we  have 
found  that  agencies  face  certain  challenges  in  implementing  the 
approach — especially  for  complex  or  major  investments.  My  testimony 
today  will  focus  on  the  particular  challenges  DHS  has  confronted. 
Specifically,  I  will  discuss  how  contract  outcomes  are  influenced  by  how 
well  DHS  components  have  defined  and  developed  contract  requirements 
and  performance  standards.  I  will  also  discuss  the  need  for  improved 
assessment  and  oversight  to  ensure  better  outcomes. 

My  statement  is  based  on  our  report  that  is  being  released  today.2  This 
report  focused  on  contracts  for  major  investments  with  complex  service 
acquisitions  at  the  Coast  Guard,  Customs  and  Border  Protection  (CBP), 
and  the  Transportation  Security  Administration  (TSA) — three  of  the  DHS 
components  reporting  among  the  highest  obligations  for  performance- 


Vor  example,  GAO,  Defense  Acquisitions:  Tailored  Approach  Needed  to  Improve  Service 
Acquisition  Outcomes,  GAO-07-20  (Washington,  D.C.:  Nov.  9,  2006). 

2GAO,  Department  of  Homeland  Security:  Better  Planning  and  Assessment  Needed  to 
Improve  Outcomes  for  Complex  Service  Acquisitions,  GAO-08-263  (Washington,  D.C.: 
Apr.  22,  2008). 
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based  service  acquisitions  in  fiscal  years  2005  and  2006.  We  conducted  this 
performance  audit  in  accordance  with  generally  accepted  government 
auditing  standards.  Those  standards  require  that  we  plan  and  perform  the 
audit  to  obtain  sufficient,  appropriate  evidence  to  provide  a  reasonable 
basis  for  our  findings  and  conclusions  based  on  our  audit  objectives.  We 
believe  that  the  evidence  obtained  provides  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives. 


Summary 


Over  the  past  several  years,  we  have  found  that  if  agency  service 
acquisitions,  including  those  that  are  performance-based,  are  not 
appropriately  planned,  structured,  and  monitored,  there  is  an  increased 
risk  that  the  government  may  receive  products  or  services  that  are  over 
budget,  delivered  late,  and  of  unacceptable  quality.  Several  prior  GAO  and 
DHS  Inspector  General  reviews  of  major  DHS  investments  using  a 
performance-based  approach  point  to  such  shortcomings.  For  the  report 
we  are  releasing  today,  we  reviewed  contracts  for  eight  major  investments 
at  the  Coast  Guard,  CBP,  and  TSA  and  found  that  all  had  outcome-oriented 
requirements — as  required  in  acquisition  regulations  and  policy.  However, 
contracts  for  four  of  these  investments  did  not  have  well-defined 
requirements,  or  a  complete  set  of  measurable  performance  standards,  or 
both  at  the  time  of  contract  award  or  start  of  work.  These  service 
contracts  experienced  cost  overruns,  schedule  delays,  or  did  not 
otherwise  meet  performance  expectations.  In  contrast,  service  contracts 
for  the  other  four  investments  had  well-defined  requirements  linked  to 
measurable  performance  standards  and  performed  within  budget  meeting 
the  standards  in  all  cases  where  contractors  had  begun  work.  DHS 
components  conducted  quality  assurance  surveillance,  and  for  the 
contracts  that  had  negative  outcomes,  surveillance  helped  to  identify 
contractor  performance  weaknesses  and  corrective  action  was  taken. 

In  managing  its  service  acquisitions,  including  those  that  are  performance- 
based,  DHS  has  faced  oversight  challenges,  including  a  lack  of  reliable 
data  and  systematic  management  reviews.  Although  contracting  and 
program  staff  at  DHS  components  told  us  that  they  used  a  performance- 
based  approach  to  the  maximum  extent  practicable,  the  department  does 
not  have  reliable  data  to  facilitate  required  reporting  or  perform 
management  assessments  of  these  acquisitions.  Our  review  of  an 
additional  138  contracts,  which  were  primarily  for  basic  services,  found 
that  about  half  of  the  contracts  coded  by  DHS  as  performance-based  had 
none  of  the  three  elements  DHS  requires:  a  performance  work  statement, 
measurable  performance  standards,  or  a  quality  assurance  surveillance 
plan.  Inaccurate  data  limit  DHS’s  visibility  over  service  acquisitions  and 
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the  department’s  ability  to  make  informed  acquisition  management 
decisions.  The  Chief  Procurement  Officer  (CPO),  who  has  responsibility 
for  departmentwide  procurement  oversight,  has  begun  some  initial  review 
of  performance-based  service  acquisitions,  but  has  not  conducted 
management  assessments  of  this  acquisition  method. 


Background 


Over  the  last  decade,  the  use  of  federal  service  contracting  has  increased 
and  now  accounts  for  over  60  percent  of  federal  procurement  dollars 
spent  annually.  A  performance-based  approach  to  federal  service 
contracting  was  introduced  during  the  1990s,  representing  a  shift  from 
specifying  the  way  in  which  contractors  should  perform  work  to 
specifying  acquisition  outcomes.  Regardless  of  the  contracting  method, 
focusing  on  outcomes  and  collaboration  among  multiple  stakeholders  in 
the  contracting  process  has  been  acknowledged  as  sound  contract 
management.  In  2000,  federal  procurement  law  established  a  performance- 
based  approach  as  the  preferred  acquisition  method  for  services.3  The 
Federal  Acquisition  Regulation  requires  all  performance-based  service 
acquisitions  to  include 

a  performance  work  statement  that  describes  outcome-oriented 
requirements  in  terms  of  results  required  rather  than  the  methods  of 
performance  of  the  work; 

measurable  performance  standards  describing  how  to  measure  contractor 
performance  in  terms  of  quality,  timeliness,  and  quantity;  and 
the  method  of  assessing  contract  performance  against  performance 
standards,  commonly  accomplished  through  the  use  of  a  quality  assurance 
surveillance  plan.4 

A  1998  Office  of  Federal  Procurement  Policy  (OFPP)  study  on 
performance-based  contracts — based  largely  on  contracts  for  basic 
services,  such  as  janitorial  or  maintenance  services — showed  that  a 


3Floyd  D.  Spence  National  Defense  Authorization  Act  for  Fiscal  Year  2001,  Pub.  L.  No.  106- 
398  §  821(a)  (2000)  required  that  the  Federal  Acquisition  Regulation  (FAR)  be  revised  to 
establish  a  preference  for  the  use  of  a  performance-based  approach  in  the  acquisition  of 
services,  which  was  done  in  FAR  37.102(a),  providing  that  performance-based  acquisition 
is  the  preferred  method  for  acquiring  services  and  generally  is  to  be  used  to  the  maximum 
extent  practicable.  In  addition,  this  act  established  a  preference  for  using  firm  fixed-price 
contracts  or  task  orders — where  a  specified  price  is  paid  regardless  of  the  contractor’s 
incurred  costs — when  using  a  performance-based  approach  for  service  acquisitions. 

4FAR  37.601;  FAR  37.602(b);  FAR  37.604.  A  fourth  element,  performance  incentives,  is 
required  where  appropriate. 
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number  of  anticipated  benefits  had  been  achieved,  including  reduced 
acquisition  costs,  increased  competition  for  contracts,  and  improved 
contractor  performance.5  However,  implementing  a  performance-based 
approach  is  often  more  difficult  for  complex  acquisitions,  such  as 
information  technology,  than  it  is  for  basic  services,  because  agencies 
begin  with  requirements  that  are  less  stable,  making  it  difficult  to  establish 
measurable  outcomes.  Such  complex  acquisitions  may  need  to  have 
requirements  and  performance  standards  continually  refined  throughout 
the  life-cycle  of  the  acquisition  for  a  contractor  to  deliver  a  valuable 
service  over  an  extended  period  of  time.  OFPP  also  has  noted  in  policy 
that  certain  types  of  services,  such  as  research  and  development,  may  not 
lend  themselves  to  outcome-oriented  requirements. 

To  encourage  agencies  to  apply  a  performance-based  approach  to  service 
acquisitions,  the  Office  of  Management  and  Budget  (OMB)  established 
governmentwide  performance  targets,  which  increased  to  50  percent  of 
eligible  service  contract  dollars  for  the  current  fiscal  year.  In  January  2007, 
the  congressionally  mandated  Acquisition  Advisory  Panel  reported  that 
performance-based  acquisition  has  not  been  fully  implemented  in  the 
federal  government,  despite  OMB  encouragement,  and  recommended  that 
OMB  adjust  the  governmentwide  target  to  reflect  individual  agency 
assessments  and  plans.6  In  May  2007,  OMB’s  OFPP  issued  a  memo 
providing  that  agencies,  at  a  minimum,  were  expected  to  meet  targets 
established  and  report  on  them  in  their  management  plans.  In  response, 
DHS’s  CPO  established  a  performance-based  target  of  25  percent  for  fiscal 
year  2007,  increasing  to  40  percent  by  fiscal  year  2010,  that  was  included  in 
DHS’s  Performance-Based  Management  Plan.  The  Acquisition  Advisory 
Panel  also  recommended  that  OFPP  issue  more  explicit  implementation 
guidance  and  create  an  “Opportunity  Assessment”  tool  to  help  agencies 
identify  when  they  should  consider  using  this  acquisition  method. 


5Office  of  Management  and  Budget,  Office  of  Federal  Procurement  Policy,  A  Report  on  the 
Performance-Based  Service  Contracting  Pilot  Project,  May  1998. 

6Report  of  the  Acquisition  Advisory  Panel  to  the  Office  of  Federal  Procurement  Policy  and 
the  United  States  Congress,  January  2007. 
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Requirements  and 
Standards  Definition 
Influence  Program 
Outcomes 


Our  work  has  found  that  performance-based  acquisitions  must  be 
appropriately  planned  and  structured  to  minimize  the  risk  of  the 
government  receiving  services  that  are  over  cost  estimates,  delivered  late, 
and  of  unacceptable  quality.7  Specifically,  we  have  emphasized  the 
importance  of  clearly  defined  requirements  to  achieving  desired  results 
and  measurable  performance  standards  to  ensuring  control  and 
accountability.  Prior  GAO  and  DHS  Inspector  General  reviews  of  complex 
DHS  investments  using  a  performance-based  approach  point  to  a  number 
of  shortcomings.  For  example,  in  June  2007,  we  reported  that  a 
performance-based  contract  for  a  DHS  financial  management  system, 
eMerge2,  lacked  clear  and  complete  requirements,  which  led  to  schedule 
delays  and  unacceptable  contractor  performance.8  Ultimately,  the  program 
was  terminated  after  a  $52  million  investment.  In  March  2007,  we  similarly 
reported  that  the  Coast  Guard’s  performance-based  contract  for  replacing 
or  modernizing  its  fleet  of  vessels  and  aircraft,  Deepwater,  had 
requirements  that  were  set  at  unrealistic  levels  and  were  frequently 
changed.9  This  resulted  in  cost  escalation,  schedule  delays,  and  reduced 
contractor  accountability.  The  DHS  Inspector  General  has  also  indicated 
numerous  opportunities  for  DHS  to  make  better  use  of  sound  practices, 
such  as  well-defined  requirements.10 


Consistent  with  our  prior  work,  definition  of  requirements  and 
performance  standards  influenced  outcomes  for  the  eight  complex 
investments  we  reviewed.  In  using  a  performance-based  approach,  sound 
contracting  practices  dictate  that  required  contract  outcomes  or 
requirements  be  well-defined,  providing  clear  descriptions  of  results  to  be 
achieved.  While  all  eight  contracts  for  these  investments  had  outcome- 
oriented  requirements,  the  requirements  were  not  always  well-defined.11 


7GAO-07-20. 

sGAO,  Homeland  Security:  Departmentwide  Integrated  Financial  Management  Systems 
Remain  a  Challenge,  GAO-07-536  (Washington,  D.C.:  June  21,  2007). 

°GAO,  Coast  Guard:  Status  of  Efforts  to  Improve  Deepwater  Program  Management  and 
Address  Operational  Challenges,  GAO-07-575T  (Washington  D.C.:  Mar.  8,  2007). 

10See  for  example,  Department  of  Homeland  Security  Inspector  General,  Major 
Management  Challenges  Facing  the  Department  of  Homeland  Security,  OIG-08-11  (Jan. 
2008),  and  Department  of  Homeland  Security  Inspector  General,  Transportation  Security 
Administration’s  Information  Technology  Managed  Services  Contract,  OIG-06-23  (Feb. 
2006). 

1  'FAR  2.101  specifically  provides  that  a  performance  work  statement  for  performance- 
based  acquisitions  describe  the  required  results  in  clear,  specific,  and  objective  terms  with 
measurable  outcomes. 
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Further,  contracts  for  half  of  the  investments  did  not  have  a  complete  set 
of  measurable  performance  standards.  Appendix  I  provides  a  summary  of 
our  analysis  of  the  requirements,  performance  standards,  and  outcomes 
for  the  eight  performance-based  contracts  for  major  investments  we 
reviewed. 

Complex  investments  with  contracts  that  did  not  have  well-defined 
requirements  or  complete  measurable  performance  standards  at  the  time 
of  contract  award  or  start  of  work  experienced  either  cost  overruns, 
schedule  delays,  or  did  not  otherwise  meet  performance  expectations.  For 
example,  contracts  for  systems  development  for  two  CBP  major 
investments  lacked  both  well-defined  requirements  and  measurable 
performance  standards  prior  to  the  start  of  work  and  both  experienced 
poor  outcomes.  The  first,  for  DHS’s  Automated  Commercial  Environment 
(ACE)  Task  Order  23  project — a  trade  software  modernization  effort — was 
originally  estimated  to  cost  $52.7  million  over  a  period  of  approximately  17 
months.12  However,  the  program  lacked  stable  requirements  at  contract 
award  and,  therefore,  could  not  establish  measurable  performance 
standards  and  valid  cost  or  schedule  baselines  for  assessing  contractor 
performance.  Software  requirements  were  added  after  contract  award, 
contributing  to  a  project  cost  increase  of  approximately  $21.1  million,  or 
40  percent,  over  the  original  estimate.  Because  some  portions  of  the  work 
were  delayed  to  better  define  requirements,  the  project  is  not  expected  to 
be  completed  until  June  2009 — about  26  months  later  than  planned. 

The  second,  Project  28  for  systems  development  for  CBP’s  Secure  Border 
Initiative  (SBInet) — a  project  to  help  secure  a  section  of  the  United  States- 
Mexico  border  using  a  surveillance  system — did  not  meet  expected 
outcomes  due  to  a  lack  of  both  well-defined  requirements  and  measurable 
performance  standards.  CBP  awarded  the  Project  28  contract  planned  as 
SBIwef  s  proof  of  concept  and  the  first  increment  of  the  fielded  SBinet 
system  before  the  overall  SBI net  operational  requirements  and  system 
specifications  were  finalized.  More  than  3  months  after  Project  28  was 
awarded,  DHS’s  Inspector  General  reported  that  CBP  had  not  properly 


12Begun  in  2001,  ACE  is  intended  to  replace  and  supplement  existing  cargo  processing 
technology  and  will  be  developed  and  deployed  in  a  series  of  increments.  The  goals  of  ACE 
include  (1)  supporting  border  security  by  enhancing  analysis  and  information  sharing  with 
other  government  agencies  and  providing  CBP  with  the  means  to  decide  before  a  shipment 
reaches  the  border  if  it  should  be  targeted  or  expedited  and  (2)  streamlining  time- 
consuming  and  labor-intensive  tasks  for  CBP  personnel  and  the  trade  community  through  a 
national  trade  account  and  single  Web-based  interface.  Task  Order  23  was  the  sole  focus  of 
our  review. 
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defined  SBInefs  operational  requirements  and  needed  to  do  so  quickly  to 
avoid  rework  of  the  contractor’s  systems  engineering.  We  found  that 
several  performance  standards  were  not  clearly  defined  to  isolate  the 
contractor’s  performance  from  that  of  CBP  employees,  making  it  difficult 
to  determine  whether  any  problems  were  due  to  the  contractor’s  system 
design,  CBP  employees,  or  both.  As  a  result,  it  was  not  clear  how  CBP 
intended  to  measure  compliance  with  the  Project  28  standard  for 
probability  of  detecting  persons  attempting  to  illegally  cross  the  border. 
Although  it  did  not  fully  meet  user  needs  and  its  design  will  not  be  used  as 
a  basis  for  future  SBfnef  development,  DHS  fully  accepted  the  project  after 
an  8-month  delay.13  In  addition,  DHS  officials  have  stated  that  much  of  the 
Project  28  system  will  be  replaced  by  new  equipment  and  software. 

Conversely,  we  found  that  contracts  with  well-defined  requirements  linked 
to  measurable  performance  standards  delivered  results  within  budget  and 
provided  quality  service.  For  example,  contracted  security  services  at  the 
San  Francisco  International  Airport  for  TSA’s  Screening  Partnership 
Program  had  well-defined  requirements,  and  all  measurable  performance 
standards  corresponded  to  contract  requirements — an  improvement  from 
our  prior  reviews  of  the  program.14  The  requirements  for  gate,  checkpoint, 
and  baggage  screening  services  clearly  stated  that  the  contractor  should 
use  technology  and  staff  to  prevent  prohibited  items  from  entering  sterile 
areas  of  the  airport  and  should  work  to  minimize  customer  complaints 
while  addressing  in  a  timely  manner  any  complaints  received.  The 
performance  standards  assessed  how  often  screeners  could  successfully 
detect  test  images  of  prohibited  items  in  checked  baggage;  the  percentage 
of  audited  records  and  inspected  equipment,  property,  and  materials  that 
were  well-kept,  operational,  and  recorded  on  maintenance  logs;  and 
whether  all  new  hires  received  the  required  training  before  assuming  their 
screening  responsibilities.  In  terms  of  expected  outcomes,  the  contractor 
achieved  a  2.2  percent  cost  underrun  during  the  first  5  months  of  the 
contract  and  exceeded  most  requirements. 


13GAO,  Secure  Border  Initiative:  Observations  on  the  Importance  of  Applying  Lessons 
Learned  to  Future  Projects,  GAO-08-508T  (Washington,  D.C.:  Feb.  27,  2008). 

14GAO,  Aviation  Security:  Preliminary  Observations  on  TSA’s  Progress  to  Allow  Airports 
to  Use  Private  Passenger  and  Baggage  Screening  Services,  GAO-05-126  (Washington,  D.C.: 
Nov.  19,  2004),  and  Aviation  Security:  Progress  Made  to  Set  Up  Program  Using  Private- 
Sector  Airport  Screeners,  but  More  Work  Remains,  GAO-06-166  (Washington,  D.C.:  Mar. 

31,  2006). 
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Unreliable  Data  and 
Lack  of  Management 
Review  Constrain 
Oversight 


In  managing  its  service  acquisitions,  including  those  that  are  performance- 
based,  DHS  has  faced  oversight  challenges,  including  a  lack  of  reliable 
data  and  systematic  management  reviews.  DHS  contracting  and  program 
representatives  told  us  that  they  use  a  performance-based  approach  to  the 
maximum  extent  practicable.  However,  DHS  does  not  have  reliable  data — 
either  from  the  Federal  Procurement  Data  System-Next  Generation 
(FPDS-NG),  the  governmentwide  database  for  procurement  spending,  or 
at  a  departmentwide  level — to  systematically  monitor  or  evaluate  or  report 
on  service  acquisitions,  including  those  that  are  performance-based. 
Reliable  data  are  essential  to  overseeing  and  assessing  the  implementation 
of  contracting  approaches,  acquisition  outcomes,  and  making  informed 
management  decisions.  Moreover,  the  Chief  Procurement  Officer  (CPO), 
who  has  responsibility  for  departmentwide  procurement  oversight,  has 
begun  some  initial  review  of  performance-based  service  acquisitions,  but 
has  not  conducted  systematic  management  assessments  of  this  acquisition 
method. 


Our  analysis  of  information  provided  by  contracting  representatives  at  the 
Coast  Guard,  CBP,  Immigration  and  Customs  Enforcement  (ICE),  and  TSA 
showed  that  about  51  percent  of  the  138  contracts  we  identified  in  FPDS- 
NG  as  performance-based  had  none  of  the  required  performance-based 
elements:  a  performance  work  statement,  measurable  performance 
standards,  and  a  method  of  assessing  contractor  performance  against 
performance  standards.  Only  42  of  the  138  contracts,  or  30  percent,  had  all 
of  the  elements,  and  about  18  percent  had  some  but  not  all  of  the  required 
performance-based  acquisition  elements  (see  table  1). 


Table  1:  Review  of  Performance-Based  Elements  on  Selected  Contracts 

Performance-based 

elements 

Coast  Guard 

Customs 
and  Border 
Protection 

Immigration 
and  Customs 
Enforcement 

Transportation 

Security 

Administration 

Total 

contracts 

Percentage 
of  total 
contracts 

All  elements 

18 

3 

0 

21 

42 

30.4 

Some  elements 

16 

0 

5 

4 

25 

18.1 

No  elements 

20 

5 

34 

12 

71 

51.5 

Total 

54 

8 

39 

37 

138 

100.0 

Source:  GAO  analysis  of  DHS  review  of  138  contracts  coded  as  performance-based  in  FPDS-NG. 


Lacking  reliable  FPDS-NG  data,  reports  on  the  use  of  performance-based 
contracts  for  eligible  service  obligations  are  likely  inaccurate.  Data 
reported  on  the  use  of  performance-based  contracts  by  service  types — 
ranging  from  basic,  such  as  janitorial  and  landscaping,  to  complex,  such  as 
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information  technology  or  systems  development — requested  by  OFPP  in 
July  2006 — are  also  likely  misleading.  The  Acquisition  Advisory  Panel  and 
DHS’s  CPO  also  have  raised  concerns  regarding  the  accuracy  of  the 
performance-based  designation  in  FPDS-NG.  The  Acquisition  Advisory 
Panel’s  2007  report  noted  from  its  review  at  10  federal  agencies  that  42 
percent  of  the  performance-based  contracts  the  panel  reviewed  had  been 
incorrectly  coded. 

Inaccurate  federal  procurement  data  is  a  long-standing  governmentwide 
concern.  Our  prior  work  and  the  work  of  the  General  Services 
Administration’s  Inspector  General  have  noted  issues  with  the  accuracy 
and  completeness  of  FPDS  and  FPDS-NG  data.15  OMB  has  stressed  the 
importance  of  submitting  timely  and  accurate  procurement  data  to  FPDS- 
NG  and  issued  memos  on  this  topic  in  August  2004  and  March  2007. 
Accurate  FPDS-NG  data  could  facilitate  the  CPO’s  departmentwide 
oversight  of  service  acquisitions,  including  those  that  are  performance- 
based. 

At  a  departmentwide  level,  CPO  representatives  responsible  for 
procurement  oversight  indicated  that  they  have  not  conducted  systematic 
assessments  including  costs,  benefits,  and  other  outcomes  of  a 
performance-based  approach.  To  improve  the  implementation  of 
performance-based  acquisitions,  CPO  representatives  established  a  work 
group  in  May  2006  to  leverage  knowledge  among  DHS  components.  They 
also  noted  that  they  are  working  with  OFPP  to  develop  a  best  practices 
guide  on  measurable  performance  standards  and  to  gather  good  examples 
of  performance-based  contracts.  In  addition,  the  CPO  has  implemented  a 
departmentwide  acquisition  oversight  program,  which  was  designed  with 
the  flexibility  to  address  specific  procurement  issues,  such  as 
performance-based  service  acquisitions,  and  is  based  on  a  series  of 
component-level  reviews.16  Some  initial  review  of  performance-based 
acquisitions  has  begun  under  this  program,  but  management  assessment 
or  evaluation  of  the  outcomes  of  this  acquisition  method  has  not  been 
conducted. 


15For  example,  GAO,  Reliability  of  Federal  Procurement  Data,  GAO-04-295R  (Washington, 
D.C.:  Dec.  30,  2003);  GAO,  Improvements  Needed  to  the  Federal  Procurement  Data 
System-Next  Generation,  GAO-05-960R  (Washington,  D.C.:  Sept.  27,  2005);  and  General 
Services  Administration  Inspector  General,  Review  of  the  Federal  Procurement  Data 
System-Next  Generation  (FPDS-NG),  Report  Number  A040127/0/T/F06016  (March  2006). 

16GAO,  Department  of  Homeland  Security:  Progress  and  Challenges  in  Implementing  the 
Department’s  Acquisition  Oversight  Plan,  GAO-07-900  (Washington,  D.C.:  June  2007). 
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Conclusion  and 
Recommendations 


Consistent  with  federal  procurement  policy,  DHS  has  emphasized  a 
performance-based  approach  to  improve  service  acquisition  outcomes. 
However,  in  keeping  with  our  prior  findings,  DHS’s  designation  of  a 
service  acquisition  as  performance-based  was  not  as  relevant  as  the 
underlying  contract  conditions.  Sound  acquisition  practices,  such  as 
clearly  defining  requirements  and  establishing  complementary  measurable 
performance  standards,  are  hallmarks  of  successful  service  acquisitions. 

In  the  cases  we  reviewed  as  well  as  in  prior  findings  where  these  key 
elements  were  lacking,  DHS  did  not  always  achieve  successful  acquisition 
outcomes.  The  report  we  are  releasing  today  recommends  that  the 
Secretary  of  Homeland  Security  take  several  actions  to  increase  DHS’s 
ability  to  achieve  improved  outcomes  for  its  service  acquisitions,  including 
those  that  are  performance-based.  These  actions  include  routinely 
assessing  requirements  for  complex  investments  to  ensure  that  they  are 
well-defined  and  developing  consistently  measurable  standards  linked  to 
those  requirements;  systematically  evaluating  outcomes  of  major 
investments  and  relevant  contracting  methods;  and  improving  the  quality 
of  FPDS-NG  data  to  facilitate  identifying  and  assessing  the  use  of  various 
contracting  methods.  DHS  generally  concurred  with  our 
recommendations,  noting  some  departmental  initiatives  under  way  to 
improve  acquisition  management.  However,  the  department’s  response  did 
not  address  how  the  CPO’s  process  and  organizational  changes  at  the 
departmental  level  will  impact  component-level  management  and 
assessment  of  complex  acquisitions  to  improve  outcomes.  Improving 
acquisition  management  has  been  an  ongoing  challenge  since  the 
department  was  established  and  requires  sustained  management  attention. 


Mr.  Chairman,  this  concludes  my  prepared  statement.  I  would  be  pleased 
to  respond  to  any  questions  that  you  or  other  members  of  the  committee 
may  have  at  this  time. 
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Appendix  I:  Key  Characteristics  of  Eight 
Performance-Based  Service  Contracts 

Major  investment  by 
component 

Service 

Well-defined 

requirements9 

Measurable 

performance 

standards'1 

Outcomes 

Coast  Guard 

Response  Boat 

Medium 

Research,  analysis, 
and  financial  and 
information 
management 

• 

• 

Contractor  submitted  all  required  documentation  on 
time;  met  project  management  quality  standards; 
and  maintained  electronic  archiving  and  restoration 
standards. 

Customs  and  Border  Protection 

Automated 

Commercial 

Environment 

Trade  systems 
software 

development  (task 
order  23) 

O 

® 

Costs  increased  by  40  percent  ($21.1  million). 

More  than  a  year  behind  schedule;  unplanned 
software  redesign. 

National  Prime 
Integration 

Maintenance  of 
equipment  used  at 
border  crossings, 
airports,  and  seaports 

• 

® 

Costs  increased  by  53  percent  ($24  million). 
Maintenance  wait  times  were  longer  than  planned. 

SBI  net 

Project  28  border 
surveillance  systems 
development  and 
fielding 

o 

® 

DHS  rejected  initial  acceptance  of  Project  28.  The 
project  was  delayed  8  months  with  final  acceptance 
in  February  2008.  DFIS  noted  that  the  contractor 
met  the  requirements,  but  the  project  did  not  fully 
meet  DFIS’s  needs  and  the  technology  will  not  be 
replicated  in  future  SBInef  development. 

Transportation  Security  Administration 

Electronic  Baggage 
Screening  Program 

Maintenance  for 
explosive  trace 
detection  machines 

• 

• 

Contractor  exceeded  the  performance  standard  for 
machine  downtime  with  a  score  1  hour  less  than 
required  and  operated  at  cost  through  the  second 
quarter  of  fiscal  year  2007. 

Screening  Partnership 
Program 

Passenger  screening 
services  at  one 
airport 

• 

• 

Contractor  exceeded  most  performance  standards; 
for  example,  threat  detection  performance  and 
false  alarm  rates  exceeded  the  quality  standards. 
Contractor  had  cost  underrun  of  2.2  percent 
($677,000). 

Secure  Flight 

Maintaining  database 
used  to  screen  airline 
passenger  data 

• 

® 

Initial  contractor  planning  reports  were  inadequate; 
system  experienced  operational  downtime; 
surveillance  reports  identified  poor  contractor 
performance.  Contractor  generally  met  time  frames 
and  delivered  within  budget. 

Transportation  Worker 
Identification 

Credential 

Issuing  identification 
credentials  to 
maritime  workers 

• 

• 

Outcomes  not  available  at  the  time  of  our  review. 

Legend:  •  contract  met  or  mostly  met  the  criteria;  ®  contract  partially  met  the  criteria;  O  contract  did 
not  meet  the  criteria. 


Source:  GAO  analysis. 
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Appendix  I:  Key  Characteristics  of  Eight 
Performance-Based  Service  Contracts 


“Well-defined  requirements  should  provide  clear  descriptions  of  results  to  be  achieved  at  the  time  of 
the  award  or  start  of  work  and  primary  requirements  should  not  change  substantially  following 
contract  award. 

bThe  set  of  measurable  performance  standards  for  a  contract  enables  the  government  to  assess  all 
aspects  of  the  contractor's  work  in  terms  of  quality,  timeliness,  and  quantity.  The  contract’s 
performance  standards  are  also  linked  to  the  requirements. 
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